Second-Order NLP Adversarial Examples

Adversarial example generation methods in NLP rely on models like language models or sentence encoders to determine if potential adversarial examples are valid. In these methods, a valid adversarial example fools the model being attacked, and is determined to be semantically or syntactically valid by a second model. Research to date has counted all such examples as errors by the attacked model. We contend that these adversarial examples may not be flaws in the attacked model, but flaws in the model that determines validity. We term such invalid inputs second-order adversarial examples. We propose the constraint robustness curve and associated metric ACCS as tools for evaluating the robustness of a constraint to second-order adversarial examples. To generate this curve, we design an adversarial attack to run directly on the semantic similarity models. We test on two constraints, the Universal Sentence Encoder (USE) and BERTScore. Our findings indicate that such second-order examples exist, but are typically less common than first-order adversarial examples in state-of-the-art models. They also indicate that USE is effective as constraint on NLP adversarial examples, while BERTScore is nearly ineffectual. Code for running the experiments in this paper is available at https://github.com/jxmorris12/second-order-adversarial-examples.Comment: 8 page

Reevaluating Adversarial Examples in Natural Language

State-of-the-art attacks on NLP models lack a shared definition of a what constitutes a successful attack. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows some linguistic constraints. We then analyze the outputs of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences.With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.Comment: 15 pages; 9 Tables; 5 Figure

Text Embeddings Reveal (Almost) As Much As Text

How much private information do text embeddings reveal about the original text? We investigate the problem of embedding \textit{inversion}, reconstructing the full text represented in dense text embeddings. We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We find that although a na\"ive model conditioned on the embedding performs poorly, a multi-step method that iteratively corrects and re-embeds text is able to recover $92\%$ of $32\text{-token}$ text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. Our code is available on Github: \href{https://github.com/jxmorris12/vec2text}{github.com/jxmorris12/vec2text}.Comment: Accepted at EMNLP 202

Tree Prompting: Efficient Task Adaptation without Fine-Tuning

Prompting language models (LMs) is the main interface for applying them to new tasks. However, for smaller LMs, prompting provides low accuracy compared to gradient-based finetuning. Tree Prompting is an approach to prompting which builds a decision tree of prompts, linking multiple LM calls together to solve a task. At inference time, each call to the LM is determined by efficiently routing the outcome of the previous call using the tree. Experiments on classification datasets show that Tree Prompting improves accuracy over competing methods and is competitive with fine-tuning. We also show that variants of Tree Prompting allow inspection of a model's decision-making process.Comment: Both first authors contributed equally; accepted to EMNLP 202

Annular electroconvection with shear

We report experiments on convection driven by a radial electrical force in suspended annular smectic A liquid crystal films. In the absence of an externally imposed azimuthal shear, a stationary one-dimensional (1D) pattern consisting of symmetric vortex pairs is formed via a supercritical transition at the onset of convection. Shearing reduces the symmetries of the base state and produces a traveling 1D pattern whose basic periodic unit is a pair of asymmetric vortices. For a sufficiently large shear, the primary bifurcation changes from supercritical to subcritical. We describe measurements of the resulting hysteresis as a function of the shear at radius ratio $\eta \sim 0.8$. This simple pattern forming system has an unusual combination of symmetries and control parameters and should be amenable to quantitative theoretical analysis.Comment: 12 preprint pages, 3 figures in 2 parts each. For more info, see http://mobydick.physics.utoronto.c

Adoption of Participatory Rural Appraisal: A Case Study from China

There are many models of technology transfer. They vary from the linear scientist-extension worker-farmers model to the integrative natural resource management model (Jiggins, 1993). International experience has shown that for small holding farmers in developing countries a farmer driven model based on participatory approaches (the Participatory Rural Appraisal (PRA) Model) is more effective and efficient

GRB Polarimetry with POET

POET (Polarimeters for Energetic Transients) represents a concept for a Small Explorer (SMEX) satellite mission, whose principal scientific goal is to understand the structure of GRB sources through sensitive X‐ray and γ‐ray polarization measurements. The payload consists of two wide field‐of‐view (FoV) instruments: a Low Energy Polarimeter (LEP) capable of polarization measurements in the energy range from 2–15 keV and a high energy polarimeter (Gamma‐Ray Polarimeter Experiment or GRAPE) that would measure polarization in the 60–500 keV energy range. The POET spacecraft provides a zenith‐pointed platform for maximizing the exposure to deep space. Spacecraft rotation provides a means of effectively dealing with any residual systematic effects in the polarization response. POET provides sufficient sensitivity and sky coverage to measure statistically significant polarization (for polarization levels in excess of 20%) for ∼80 GRBs in a two‐year mission. High energy polarization data would also be obtained for SGRs, solar flares, pulsars and other sources of astronomical interest